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Abstract 


The literacy proficiency development is one of concerns in education generally, so is in mathematics 
education as well. The growth of literacy proficiency is one of the issues in education, because it is very 
important to problem solving skills in students’ real life. This research aimed to describe the growth 
of lower secondary school and upper secondary school students’ mathematical literacy proficiency 
in Yogyakarta Special Region province, Indonesia. Three mathematical literacy proficiency aspects 
were examined in the research, namely content, process and context. It was an exploratory descriptive 
research with cross-sectional type research design. The population was 1,001 lower secondary school 
and upper-secondary school students ranging between 13 and 16 years old. They were selected using the 
combination of stratified and the cluster random sampling technique. A test consisting of 30 items, was 
adopted from the existing PISA test items used to collect the data in the research. The main data analysis 
was conducted by estimating students’ ability through the item-response theory approach. The results 
showed that the mathematical literacy proficiency of the students based on content, context, and process 
was still low. In the content and context domain, there was progress from &" grade to 9", from 9" grade 
to 10" grade. In the process domain, the development of students’ abilities on formulate showed relatively 
the same results for 8", 9", and 10" were around 500, and in the employ and interpret process domain, 
there was a development of abilities from 8" to 9", and from 9" to 10" grade. 

Keywords: mathematics literacy proficiency development, domains based on PISA study. 


Introduction 


Advances in Information and Communication Technology (ICT) have introduced new 
changes and challenges. Education is expected to prepare individuals with skills to deal with 
these changes and challenges. Many countries have realized the need to equip their young 
generation through education with multiple competencies. One of the students’ competences 
that has become an international issue is literacy proficiency. Literacy is one of the main 
competencies that learners need to face the challenges of the 21‘ Century (Drew, 2012). The 
traditional definition of literacy is the ability to read and write a short simple statement about 
individual’s life (Krasny, 2013, p. 14). Today, literacy does not only evolve into being able 
to read, but also about being intellectual, and knowing how to research and solve complex 
problems. Therefore, literacy is fundamental for individuals to be able to participate in society 
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and achieve their goals in work and life (UNESCO, 2015, pp. 136—137). In addition, literacy 
also has an effect on all cognitive domains (Matute et al., 2012, p. 124). 

Literacy proficiency has been assessed in Programme for International Students’ 
Assessment (PISA) that was initiated by the Organization for Economic Co-operation and 
Development (OECD). The objectives of PISA are to assess the students’ knowledge and 
skills in the real world and to prepare them with the long-life learning and the community 
participation (Stacey, 2011, p. 105). The result from PISA might be used by the government 
to, for example, monitor the educational system (Retnawati & Wulandari, 2019; Stacey, 2011). 
There are three aspects of literacy proficiency that are measured namely as reading literacy, 
mathematical literacy, and scientific literacy proficiency (OECD, 2017). 

Mathematical literacy is related to the individual proficiency in formulating, identifying, 
understanding and implementing the mathematical foundations in multiple contexts that an 
individual needs in daily life (Ojose, 2011, p. 90). It is also needed to verify the solutions 
to problems that have been created (Hillman, 2014). Mathematical literacy is important for 
students’ competency to read, write, and speak about mathematics (Atsnan, Gazali, & Narek1, 
2018; Casey, 2013; Hillman, 2014). The result of many studies showed that mathematical 
literacy is affected by some factors, e.g. school-level characteristics (Chowa, Masa, Ramos, & 
Ansong, 2015; Kartianom & Ndayizeye, 2017), and teachers’ behaviors (Magen-Nagar, 2016, 
pp. 318-319) which are related to the implementation of learning in the classroom. Another 
factor arising from the students themselves, includes mathematics interest and self-concept 
(Uysal, 2015, p. 1670), grade level, gender (Magen-Nagar, 2016, p. 318), time allocated for 
learning mathematics (Savas, Tas, & Duru, 2010, p. 113), learning facility at home (Turkan, 
Uner, & Alci, 2015, p. 359), as well as the economic, social and cultural status (Kartianom & 
Ndayizeye, 2017Stacey, 2011). 

The mathematical literacy proficiency involves some aspects of mathematical thinking 
including reasoning, modelling, making connections between idea (NCCA, 2012, p. 8), 
mathematical concept, mathematical procedure, and mathematical fact. These aspects are 
central in explaining and predicting a phenomenon by emphasizing the competencies of 
process, content and context (OECD, 2006). The domain of the content to assess includes the 
Change and Relationship (CR), Shape and Space (SS), Quantity (QNT), and Uncertainty and 
Data (UD). The next domain is the context that is related to Personal (PER), Occupational 
(OCCP), Societal (SOC), and Scientific (SC). The process competencies in the mathematical 
literacy proficiency are to formulate (FRM), employ (EMP), and interpret (INT). 

In order to measure the context competencies, a researcher should implement several 
types of test items. According to Shiel, Perkins, Close, and Oldham (2007), the test item designs 
for the PISA assessment format are the traditional multiple-choice items, the complex multiple- 
choice items, the closed-constructed response items, the short-response items and the open- 
constructed response items.” After the students’ responses have been analyzed, the individual 
capacity was classified into 7 levels, starting from below Level | to Level 6. The descriptions 
of students’ capacity are at each level using the PISA classification. 


Research Problem 


Based on the results of a study conducted by PISA from 2000 to 2015, it was found 
that only few students were able to reach level 4 or above. Meanwhile, most of them are still 
below level 2. Some parties claimed that the low level of students’ literacy proficiency showed 
the failure of education system organized by the government. Teachers are considered still 
not successful in training student literacy. But the other consider that the results of the PISA 
study cannot be used as a basis to justify the quality of education in a country. The things that 
are debated are related to sampling, context, and differences in curriculum in each country. 
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Retnawati and Wulandari (2019) described literacy proficiency development, but their research 
had not explained the development of literacy proficiency based on its domain, namely content, 
context, and process. 

Although the correlation between the results of the PISA and the quality of education 
is still debated, the results of PISA provide an overview of the growth mathematical literacy 
proficiency of students in Indonesia in recent years. However, PISA results have not been able 
to explain the mathematical literacy proficiency of students based on its domain. In addition, 
students’ mathematical abilities which also influence the literacy proficiency of students were 
very diverse (Balitbang Kemdikbud RI, 2014, 2015). 

By utilizing the development of literacy proficiency in detail, educators can find out in 
which part of the students’ development that needs to be improved. Likewise, further research 
can be carried out, in order to ensure optimal proficiency development, for example, support 
for teaching materials and increasing the ability of teachers to practice literacy. Based on this 
problem, a research to describe the growth of mathematical literacy based on its domain was 
needed. 


Research Focus 


The focus of this research was the growth of mathematical literacy proficiency. Based on 
the PISA international study, the literacy proficiency could be determined in three domains, that 
were content, process and context (OECD, 2006). 


Research Aims 


The aims of this research were to describe the growth of mathematical literacy proficiency 
of students based on the domains of literacy, that are content, process and context domain. 


Research Questions 


The research questions were: 

1. How was the growth of literacy mathematics based on content domain? 
2. How was the growth of literacy mathematics based on context domain? 
3. How was the growth of literacy mathematics based on process domain? 


Research Methodology 
Research Design 


The research was an exploratory descriptive research with cross-sectional type research 
design. Within the research, the researcher would like to describe the growth of mathematical 
literacy proficiency of the students in the 8th, 9th and 10th grade, especially the trend in 
content, context, and process literacy. This research was conducted with the stages of preparing 
instrument by adapting items released PISA, proving the validity and estimating reliability, 
conduct tests to students, estimating the ability of students in the content, context, and process 
of mathematical literacy using the international item parameters, then present the results of the 
analysis to describe the trend of development of students’ mathematical literacy proficiency 
in content, context, and process domain. The data collecting was conducted in 2015, and data 
analysis was conducted in 2016-2017. 
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Population and Sample 


The population of the research was about 125,000 lower secondary school and secondary 
school students, the students ranging between 13 and 15 in the lower secondary schools and 
the students ranging between 15 and 16 years old in the upper secondary schools in Yogyakarta 
Special Region Province in Indonesia. A combination of the stratified random sampling technique 
and the cluster random sampling technique was used to select the research participants. The 
sample comprised the 8" and 9" grade students from the lower secondary school and upper 
secondary school degree and the 10" grade students from the upper secondary school degree in 
Yogyakarta Special Region. These students came from three different levels of schools, namely 
the high-performance, the moderate-performance and the low-performance category. This 
categorization was made based on the scores in the Mathematics National Examination. A total 
of 1,001 students participated in the research, 464 male and 537 female, 155 students were in 
their 8" grade (13-14 years old), 386 students were in their 9th grade (14-15 years old), and 460 
students were in their 10th grade (15-16 years old). The sample size determined by formula to 
estimate 95% confidence interval for mean of students’ literacy proficiency, using error 6.5 and 
standard deviation 100, and got minimum sample size 909.25. 

When data collecting was conducted, researchers informed the teachers and students that 
the test was conducted only for research. All of identities about students, teachers, and school 
were coded. The results of the test were not used for any decision about students. 


Instrument 


A test consisting of 30 items was used to collect the data in the research. The test items 
were adopted from the existing PISA test items, i.e. PISA 2003, 2007 and 2011. These items 
were translated into Bahasa Indonesia (Indonesian language) and the contexts were also 
adjusted to correspond Indonesian contexts. These test items were developed and validated by 
Wulandari (Jailani, Retnawati, Musfiqi, & Wulandari, 2015). The validity of the test instrument 
was examined through the content validity that was conducted in order to identify the relevance 
and the representativeness of the instrument toward the domain under assessment. It involved 
consulting the test instrument to experts (professional judgments) in relation to the domains 
of content and context and the domain of process in the PISA test-item model. The domain of 
test item content includes the four contents (QNT, CR, SS, and UD). The content validity was 
also examined to identify the coverage and the relevance of the test items to the domain of 
context (PER, SSC, OCCP, and SSC), and to the domain of process (FRM, EMP, and INT). The 
experts also provided feedbacks regarding the material truth, the composition of substances in 
each domain, the test item readability and the relevance between the test item context and the 
students in Indonesia. 

Cronbach’s @ was used as a measure of the reliability of the test that took the form of 
essay or multiple choice with the dichotomous data. The index of reliability was .707 and the 
SEM was equal to 2.81. The Cronbach’s a indicated the internal consistency at this level is 
considered reasonably high. Based on the SEM score, the researchers would like to imply that 
if the PISA test 1tem model had been administered again then the score that the students would 
attain would be from XT — 2.81 until XT + 2.81. 


Data Analysis 
The students’ abilities were estimated first by implementing the Rasch model in the 


unidimensional item response theory. The abilities were used to analyze the growth of literacy 
proficiency. The estimation was conducted to utilize the item parameters that had been equalized 
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into the international study test item with the concordance model for the linking score. The 
concordance was conducted by implementing the Mean and Mean Method. The researcher 
subsequently interpreted the inter-year ability literacy proficiency, especially in content, 
context, and process literacy. 


Table 1 
Adjustment of ability parameter into the PISA international parameter 


Mean of Difficulty Parameter 


Domai Sub- ——————_—_——————___—_—_— — Equation of Ability Scale 
omain . ; Beta . 
Domain : This Conversion 
International Research 
Research 
CR 0.935 -0.7005 1.6355 a&=4,+ 1.6355 
QNT 0.2599 0.4768 -0.2169 é& = 4, — 0.2169 
Content | 
55 —0.3173 -0.1807 -0.1367 6 = 4, — 0.1367 
UD —0.7122 -1.04188 0.3297 é& = 4, + 0.3297 
OCCP —0.6751 -1.7395 1.0644 é& = 6, + 1.0644 
Context PER 0.5334 0.3110 0.16214 é& = 4, + 0.1621 
oC 1.441 -0.5215 1.9625 é&=64,+1.9625 
SOC —1.3221 -0.3367 -0.9855 é = 4, — 0.9855 
EMP —0.86963 -0.92664 0.0570 a&=6,+ 0.0570 
Process FRM 0.036693 -0.298 0.3347 é& = 4, + 0.3347 
INT 0.9454 0.4944 0.451 é& = 6,+0.451 


The steps of data analysis were as follows: (1) Estimating the item parameters and the 
ability parameters by operating the Rasch model proposed by Masters (2010). It was applied 
to both the students’ response from the multiple-choice test items and the dichotomous and 
polytomous constructed response test item. CONQUEST program was used in the analysis (Wu, 
Adams, & Wilson, 1997) with the calibration concurrent model for the 8", 9" and 10" grade; (2) 
Adjusting the test item parameters to the international test item parameters by means of Mean 
and Mean method (Hambleton & Swaminathan, 1985). It was conducted until the researcher 
attained the adjustment of the item parameter to the international scale for the 8", 9" and 10" 
grade. The results of the adjustment are presented in Table 1; (3) Implementing the modification 
equation from the second step in order to adjust the capacity parameter for each class; (4) 
performing a descriptive analysis in order to present the growth of the mathematical literacy 
proficiency of the students for the 8", 9 and 10" grade; and (5) categorizing the participants’ 
ability in accordance with the data analysis within the PISA model for each classroom by using 
the results of participants’ capacity estimation. 

The participants’ ability in accordance with the data analysis within the PISA model, 
the student ability scale should be transformed to a mean that was equal to 500 and the 
standard deviation that was equal to 100, minimum 0 and maximum 1,000. The results of the 
transformation were then categorized into 7 levels that consisted of Below Level 1, Level 1, 
Level 2, Level 3, Level 4, Level 5 and Level 6 in accordance to the Technical Report (OECD, 
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2016). Based on the results of transformation and the categorization of literacy proficiency, the 
researcher subsequently monitored the development trend. The growth of the mathematical 
literacy proficiency described by referring to the standard and by considering the aspect of 
literacy content, context and process. 


Research Results 
Literacy Based on the Content 


The mathematical literacy proficiency of the 10" grade students was better than that of 
the 8" and 9" grade students’, particularly for the content of QNT, SS, and UD. The complete 
result is shown in Figure |. This finding showed that there was an improvement on the 8", 9" 
and 10 grade students’ mathematical literacy for the content of QNT, SS, and UD. On the 
contrary, the 8" grade students had the highest mathematical literacy proficiency in comparison 
to the 9" and 10™ grade students for the content of CR. This finding showed that there was a 
decreasing on the students’ mathematical literacy proficiency for the content of CR. 


Figure 1 
The mathematical literacy proficiency based on content 
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The growth of the students’ mathematical literacy in the low school level improved 
among the 8", 9" and 10" grade for the content of QNT, SS, UD. This result is shown in Figure 
2. However, the highest score in CR was obtained by the 8" grade students. The growth of 
students’ mathematical literacy proficiency in the moderate school tended to be unstable which 
implied the increasing and the decreasing proficiency in accordance with the students’ grade. 
For example, the Figure 2 shows that the literacy proficiency for the content of QNT increased 
from the 8" grade to the 9" grade but decreased from the 9" grade to the 10" grade. Similar 
result was obtained for the quantity content. In the meantime, the students’ mathematical 
literacy proficiency in the high school level had improved along with the students’ grade for all 
of the PISA contents that had been administered. 
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Figure 2 
The mathematical literacy proficiency based on content domain, grade and school level 
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Overall, there was an improvement on the students’ mathematical literacy in the 9" and 
10" grade in accordance with the school level. The higher the school level was, the higher the 
mathematical literacy proficiency that the students attained. However, for the 8™ grade students, 
the moderate school level attained the highest score in comparison to the high school level. The 
possible reason was that the 8™ grade was the most prominent. 


Figure 3 
The mathematical literacy on the content of QNT 
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There was an improvement of the students’ mathematical literacy and that the students 
achieved Level 1 to Level 6 for the content of QNT. There were 10% of the 9" and 10" 
grade students who had been able to achieve the Level 6. However, in general the students’ 
mathematical literacy for the content of QNT was low because most of the students only 
achieved the following category: Below Level | — Level 3. Furthermore, there were only 20% 
students who achieved Level 4 — Level 6. These results are shown in Figure 3. 
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Figure 4 
The level of mathematical literacy proficiency on the content of CR 
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Figure 4 shows that base on the students’ grade, the mathematical literacy proficiency for 
the content of a CR tended to be unstable. However, there were many 8" grade students who 
were in the “Level 6” category compared to the 9" and 10" grade students. This result might be 
due to the fact that the learning material of CR is taught to students in their 8" grade. 


Figure 5 
The level of mathematical literacy proficiency on the content of SS 
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Another content that became the focus of assessment in the mathematical literacy 
proficiency was the content of SS. Figure 5 shows that, overall, the mathematical literacy 
proficiency of students in their 8" grade, 9" grade, and 10" improved along with the grade level. 
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However, in general, the students’ mathematical literacy proficiency for the content of SS was 
low because most of the students achieved the below level 1—level 3 category. Figure 6 shows 
that the growth of mathematical literacy proficiency for the content of UD from the 8" grade 
students until the 10 grade students improved. 


Figure 6 
The mathematical literacy proficiency on the content of UD 
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Literacy Proficiency Based on the Context 


The contexts that had been implemented in the mathematical literacy proficiency were in 
accordance with the standards that had been implemented in the PISA and the contexts included 
the use of Mathematics in the personal life (PER), social life (SOC), occupation (OCC), and 
science (SC). Being adjusted to the level of ability that became the standards of PISA, the 
students’ mathematical literacy proficiency was also differentiated for each domain. 

Based on the Figure 7, the researchers found that the students’ mathematical literacy 
proficiency improved in all contexts. The improvement was in accordance with the students’ 
grade level. The findings showed that the students’ mathematical literacy proficiency in the 
Province of Yogyakarta Special Region improved in accordance to the grade level. 

However, in this case the 8" and the 9™ grade students achieved the highest score for 
the SC context while the 10" grade students achieved the highest score for the OCC context. 
Meanwhile, for the PER and SOC context, the students achieved lower score than the other 
two contexts (OCC, and SC). The reason was that the PER and SOC context test items were 
designed under the process of interpreting, implementing and evaluating the mathematical 
results (the third domain of process) and under the indicator of drawing the conclusion on the 
mathematical results toward the contextual problems and of evaluating and providing logical 
reasons or arguments toward the mathematical results that had been attained. In addition, the 
students’ ability in this domain of process was lower than their ability in the other two domains 
of process (FRM, and EMP). 
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Figure 7 
The mathematical literacy proficiency based on the Context 
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Figure 8 shows that the students’ mathematical literacy proficiency in the low, moderate, 
and the high school level increased from the 8" grade to the 9" grade and from the 9" to the 
10" grade for the OCC and SC context. On the contrary, the students’ mathematical literacy 
proficiency for the PER and SOC context in the low and moderate school level decreased from 
the 8" to the 9" grade but increased from the 9" to the 10" grade. Then, for all contexts the 
students’ mathematical literacy proficiency increased from the 8" grade to the 9" grade and 
from the 9" grade to the 10" grade. In general, the 9" and the 10™ grade students’ mathematical 
literacy proficiency increased for all contexts in the low, moderate and high-level schools. The 
reason was that the model level school that had been sampled was the best moderate level 
school. 


Figure 8 
Mathematical literacy proficiency based on the context, grade and the school level 





800 





700 





fs 





500 = & 





A00 i Occupational 


Score 


=: Personal 


3 


Chee eee Eee EE OS Ee Oe ee ee a ee ee ee oe ++ 0 09 
tees thee 6 OO Oe ey oe «oy 
; ; 


4 


eee deeeeh bade 


GE 
PR ciscccacneccsecc re 


300 


lll Societal 
200 


# Scientific 


eee wees ladle ee eee ee 
Cee 


@ 


100 


eee CREE CEES SS EMME ESE Eee ee eee 
4 
SEECSES CSS ECCS CCE E CSET CSET ESSEC SCC EE: 


i 
Boe sesso secs SCA Se ee eC CGC OGG 


4 


Gi 


lo [Receeeeeceee 


> 
© 
co 
eS 
© 
Ne) 
j= 
i) 


Midlle High 
Class & School Level 


° 
= 


ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/20.78.80 


PROBLEMS 
OF EDUCATION 


IN THE 21" CENTURY 
Vol. 78, No. 1, 2020 





Jailani JAILANI, Heri RETNAWATI, Nidya F. WULANDARI, Hasan DJIDU. Mathematical literacy proficiency development based on 
content, context, and process 


Subsequently, the growth of the students’ mathematical literacy proficiency based on 
each context is shown in Figure 9. The growth of grade 8 to 10 students’ mathematical literacy 
proficiency for the OCC context. The OCC context was related to the students’ life in the 
school or in the working environment. For the OCC context in the 8" grade, the percentage 
showed that the 8" grade students’ ability in the Province of Yogyakarta Special Region was 
equally distributed from the “Below Level 1” category until the “Level 5” category for the 
OCC context; most of the students were in “Below Level 1” category and there were very few 
students or there were almost none of the students who were in “Level 6” category. 


Figure 9 
The Level of mathematical literacy proficiency on the OCC context 
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The results shown in Figure 9 suggest that the improvement between the 9" grade and the 
10" grade students’ mathematical literacy proficiency was almost similar. The number of the 9" 
grade and the 10" grade students who were in “Below Level 1” category were very few or near 
0% and there were less than 5% of the students who were in “Level 1” category. The number 
of the 10" grade students who were in “Level 5” and the “Level 6” category was higher than 
that of the 9" grade students. The Figure showed that based on the OCC context the students’ 
mathematical literacy proficiency improved from the 8" grade to the 9" grade and from the 9" 
grade to the 10" grade in accordance with the students’ grade level. 

The domain of PER context had direct relationship to the students’ daily activities. In 
the daily activities, the students definitely encountered the personal problems that demanded 
immediate solutions. The growth of the students’ mathematical literacy proficiency from the 8" 
grade to the 10" grade students is displayed in Figure 10. 
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Figure 10 
The level of mathematical literacy proficiency on the per context 
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Figure 10 shows that for the PER context 30% of the 8", the 9" and the 10" grade students 
were in “Below Level 1” category and 10% of the 8", the 9" and the 10" grade students were 
in “Level 1” category. In addition, still based on the above Figure 10 it was apparent that the 
students’ mathematical literacy proficiency improved from the 8" grade to the 9" grade and 
from the 9" grade to the 10" grade. The improvement was found in the decreasing number of 
the 9" and the 10" grade students who were in “Below Level 1” category and the increasing 
number of the students who were in “Level 2,” “Level 3,” “Level 4” and “Level 5” category. 
However, in overall the students’ mathematical literacy proficiency for the PER context was 
still low because more than 70% of the students were still in “Below Level 1,” “Level 1,” 
“Level 2” and “Level 3” categories and around 20% of the students were in “Level 4,” “Level 


5” and “Level 6” categories. 


Figure 11 
The level of mathematical literacy proficiency on the soc context 
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The SOC context was related to the use of mathematical knowledge in the SOC life 
and the wider neighborhood in the daily life. Figure 11 shows that the growth of the students’ 
mathematical literacy proficiency was based on the SOC context. Figure 11 suggests that for 
the SOC context, most of the students in grade 8 to 10 were in “Below Level 1” category. The 
percentage above showed that based on the SOC context the students’ mathematical literacy 
proficiency improved from the 8" grade to the 9" grade and from the 9" grade to the 10" grade. 
The improvement was found in the decreasing number of 9" grade and 10" grade students who 
were in “Below Level 1” and “Level 1” category and the increasing number of the students who 
were in “Level 2” category and above. However, in general the students’ mathematical literacy 
proficiency for the SOC context was still low since most of the students were in “Below Level 
1,” “Level 1,” “Level 2” and “Level 3” categories. 

The SC context was related to the scientific activities that were more abstract and that 
demanded theoretical mastery and understanding in performing the mathematical problem 
solution (see Figure 12). For the SC context, Figure 12 shows that the students’ mathematical 
literacy proficiency improved from the 8" grade to the 9" grade and from the 9" grade to the 
10" grade in accordance with the students’ grade level. In addition, for the SC context more than 
50% of the students in the Province of Yogyakarta Special Region achieved the “Level 4,” the 
“Level 5” and the “Level 6” category. 


Figure 12 
The level of mathematical literacy proficiency on the SC context 
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Literacy Proficiency Based on the Process 
The students’ literacy proficiency was also classified into each domain of process. The 


mapping of the 8", 9" and 10" grade students’ mathematical literacy proficiency in the domain 
of process is presented in Figure 13. 
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Figure 13 
Mathematical literacy proficiency based on grade and process 
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The domain of process in formulating the mathematical situations (formulate) the 
mathematical literacy proficiency of 8", 9" and 10" grade of the students improved along with their 
grade level shown in Figure 13. Similar finding was also found in the domain of employing the 
mathematical concepts, facts, procedures and reasoning (EMP), and interpreting, implementing 
and evaluating the mathematical results (INT). However, the students’ mathematical literacy 
proficiency in the process of employing the mathematical concepts, facts, procedures and 
reasoning was lower than that in the process of formulating the mathematical situations. 
Similarly, the students’ mathematical literacy proficiency in the process of interpreting, 
implementing and evaluating the mathematical results was the lowest compared to the other 
two domains of process. These findings implied that the students were likely able to employ the 
mathematical concepts, facts, procedures and reasoning appropriately if they could formulate 
the mathematical situations. However, there were also some students who might formulate 
the mathematical situations, but they were not able to solve the mathematical situations by 
employing the mathematical concepts, facts, procedures and reasoning. The students might 
also interpret, implement and evaluate the mathematical situations appropriately if they were 
able to formulate the mathematical situations and to employ the mathematical concepts, facts, 
procedures and reasoning appropriately. 
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Figure 14 
The mathematical literacy proficiency based on process and school level 
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In addition to the domain of process, the growth of the students’ mathematical literacy 
observed by the school level (1.e., low, moderate and high). The growth of the students’ 
mathematical literacy proficiency for the domain of process in each school level is presented 
in Figure 14. The result in Figure 14 shows that the growth of the 8™ grade students had 
improved from the low-level school to the moderate-level school. However, the growth of the 
literacy proficiency had decreased in the high-level school in terms of the domain of process. In 
overall, mathematical literacy proficiency of the students had improved in accordance with the 
increasing school level and quality. 


Figure 15 
The mathematical literacy proficiency to formulate 
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Then, the growth of the students’ mathematical literacy proficiency was examined under 
each domain of process. The first domain was to formulate the mathematical situations is shown 
in Figure 15. The second domain was to employ the mathematical concepts, facts, procedures 
and reasoning. The growth of the students’ mathematical literacy proficiency from the 8" grade 
until the 9" grade in the second domain is shown in Figure 16. 
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Figure 16 
The mathematical literacy proficiency to employ 
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These findings showed that there had been improvement of the students’ mathematical 
literacy proficiency in accordance with their grade level. The improvement can be seen in the 
decreasing percentage of the 9" grade and the 10™ grade students who were in “Below Level 1” 
category and the increasing percentage of the students who were in “Level 6” category. 

Subsequently, the researchers examined the students’ mathematical literacy proficiency 
based on the third domain of process, namely, to formulate, to employ, and to interpret the 
mathematical results. Figure 17 presents the mathematical literacy proficiency of the students’ 
in the Province of Yogyakarta Special Region on the third domain of process. 


Figure 17 
The mathematical literacy proficiency to interpret 
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The percentage showed that there was improvement on the students’ mathematical 
literacy proficiency from the 8" grade to the 9" grade and from the 9" grade to the 10™ grade 
although there were few students in “Level 5” and “Level 6”. There is a decrease in the number 
of grade 9 students and 10" grade students who were in “Below Level 1” category and there 
are increasing number of students who were in “Level 4,” “Level 5” and “Level 6” category. 
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Discussion 


The results of this research showed that there was an improvement of mathematical 
literacy proficiency of the students from the 8" grade to the 9" grade and to the 10" grade. The 
higher the grade was, the higher the students’ achievement in the score of mathematical literacy 
proficiency. These results indicate that the achievement level of the students’ mathematical 
literacy proficiency was also influenced by the grade. However, the influence of age or year level 
was not significant. This result is in accordance with the work of Magen-Nagar (2016, p. 318); 
Jabor, Machtmes, Kungu, Buntat, and Nordin (2011), and also Thoren, Heinig, and Brunner 
(2016) who found that age and school level influenced the students’ literacy proficiency . The 
reason is that the mathematics learning material learned by students in grades 9 and 10 as stated 
in the standards of the Indonesian curriculum is more comprehensive than students in grades 8 
did. As a result, the higher the students’ grade was the more learning materials that supported 
the improvement of mathematical literacy would be. 

In relation to the content of mathematical literacy there was a tendency that the literacy 
within the content of CR and UD had been higher in comparison to the content of QNT and 
the content of SS. Although students’ literacy in numbers, geometry, increased from grade 8 to 
10, the students’ ability was still in the low category. It can be seen from the small number of 
students in level 4 to 6. In numbers, most of the students in grade 8 to 10 could not achieve level 
4. All students in year 8 could not even achieve level 2. This result shows that students in grade 
8 found difficulties in formulating information in most of the test 1tems. They could only answer 
questions related to geometry and numbers that have been clearly defined. The students’ low 
ability in the geometrical content indicated their low ability in spatial skills. This is in line with 
previous studies, e.g. Hannafin, Truxaw, Vermillion, and Liu (2008), and Novak and Tassell 
(2017) that spatial abilities are directly related to mathematics ability, particularly in geometry. 

The student’s literacy in algebra was not significantly different among students in their 
year 8 to 10. This is supported by the finding from previous research by Eze, Ezenwafor, and 
Obi (2015, p. 99); Josiah and Adejoke (2014, p. 475) that students age is not a significant 
correlate of students’ algebra skills. However, student’s literacy in algebra is better than their 
literacy in numbers, geometry and uncertainty dan statistics. Most of the students’ scores were 
above 669 (level 6). The possible reason is that algebra has been taught to students in year 8 
than those in year 9 and 10. 

Concerning achievement related to the process of formulating mathematical situations, 
of employing the mathematical concepts, facts, procedures and reasoning and of interpreting, 
implementing and evaluating the mathematical results, there was an improvement in the 
students’ mathematical literacy from the 8" to the 9™ grade and from the 9" to the 10" grade 
in accordance with the students’ grade level. Meanwhile, in overall the students’ ability in 
the process of employing the mathematical concepts, facts, procedures and reasoning was 
lower than their ability in the process of formulating the mathematical solutions. Similarly, the 
students’ score of mathematical literacy in the process of interpreting, employing and evaluating 
the mathematical results was the lowest one in comparison to their scores in the other two 
domains of process. These findings showed that the students were likely able to employ the 
mathematical concepts, facts, procedures and reasoning appropriately if they could formulate 
the mathematical problems. The students’ low ability in the domain of interpreting shows that 
they were not able to infer, apply, and evaluate problem solutions. Similarly, Tambychik and 
Meerah (2010, p. 150) also found that students found difficulties during making meaningful 
connection in the problem. Furthermore, Jupri, Drijvers, and Heuvel-Panhuizen (2014, p. 51); 
Hadi, Retnawati, Munadi, Apino, and Wulandari (2018) also found that students experienced 
difficulties in solving problems due to their inability in applying the reverse strategy as a step in 
verifying solutions to the problems before they come to a conclusion. 
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For the achievement related to the context, the students’ score of mathematical literacy in 
PER and SOC context was lower than their scores in the OCC and the SC context. The reason 
was that within the learning process, the mathematics learning materials directed the students to 
understand and to master the knowledge as a preparation to pursue higher level education or to 
find a job. As a result, PER and SOC context had rarely been implemented in the mathematics 
learning materials. This made the students find difficulties in answering questions that used 
the contexts. This is in line with the studies by Abdullah, Abidin, and Ali (2015, p. 140), and 
Lieven, Fien, & Erik (2015) when they had to answer questions related to contexts that they 
have not learned before. Jailani, Sugiman, and Apino (2017) suggests the need to integrate 
various contexts in the learning and teaching process. 

The students’ achievement score in mathematical literacy was influenced by the school 
level. There was correlation between the scores in mathematical literacy and achievement in 
national examination. The higher the school performance in national exams, the higher the 
achievements of its students of the school in mathematical literacy ability. The good input of 
the students, learning process and learning achievement were usually heard by school that had 
high achievement. These findings had been in accordance with the findings from Bohlmann and 
Pretorius (2008), and also Chowa et al. (2015) that school-level characteristics affect academic 
achievement. There were many factors which influenced the students’ mathematical literacy 
ability, the score in the content of CR and of UD was higher than that in the content of QNT and 
of SS both based on the school grade and the school level. The possible factor was the material 
content in every grade. The content of CR and of uncertainty dominated the contents in the 8" 
grade and the 9™ grade in Indonesian Curriculum. 

The research results showed that the students’ mathematical literacy was unsatisfying. 
It should get attention from government, teacher, and researcher. The mathematics teaching 
quality, including process of planning, implementing and assessing learning outcomes should 
support the students’ mathematical literacy proficiency. Although the quality of schools affects 
students’ ability (Chowa et al., 2015), but the quality of teaching is the most important key 
to improve the students’ achievement in mathematical literacy (Retnawati, Djidu, Kartianom, 
Apino, & Anazifa, 2018). The improvement toward the learning quality and the learning 
assessment can be pursued through the integration of the literacy into the mathematics learning 
process (Hillman, 2014, p. 403), and also to the other subjects. Besides that, mathematics 
education programs should be developed to help students be able to make real life connections 
(Apino & Retnawati, 2017; Djidu & Retnawati, 2018; Yavuz, Ilgiin Dibek, & Yalcin, 2017). 
Therefore, further studies are necessary to determine the strategies that can be used to train the 
students’ mathematical literacy, and also the development of teaching sets to teach mathematical 
literacy. 


Conclusions and Implications 


In the content domain, specifically Shape and Space (SS), Quantity (QNT), and 
Uncertainty and Data (UD), there was progress from 8" grade to 9", from 9" grade to 10" grade, 
but in Change and Relationship (CR), the ability of students from 8" grade was higher than 
students from 9" grade and 10" grade. The percentage of students whose literacy proficiency 
was in categories 1-6 in 9" grade and 10" grade tends to be higher than in 8" grade, and for 
8" grade was dominant in below level 1. For CR and UD, the average ability of students was 
approaching 600. In the context domain, the literacy ability of students in occupational and 
scientific contexts showed the average of students’ ability in 8", 9", and 10™ grade was relatively 
similar. In the personal and societal context domains, the ability of students in grades 8", 9", 
and 10" was relatively the same. For the 8" grade, students’ mathematical literacy abilities 
related to occupational and scientific contexts were already above the international average in 
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grades 9 and 10 that met 600. In the process domain, the development of students’ abilities on 
formulate (FRM) showed relatively the same results for 8", 9", and 10™ were around 500. In the 
employ and interpret process domain, there was a development of abilities from 8" to 9", and 
from 9" to 10" grade. 

Based on the results of these studies, some further research can be conducted related to 
this research. The development of mathematical literacy abilities based on content, contexts 
and processes domains that have not been as expected, needs to be determined. The decreasing 
scores of students’ literacy skills from 8th to 9" grade, from 9" to 10" grade also need to be 
known as the contributing factors. The results of students’ literacy scores in the domain of 
content, context and processes that have not reached optimal scores, efforts to improve the 
quality of mathematics learning need to be done. Research related to improve teaching and 
learning that train mathematical literacy competencies, especially related to each mathematical 
literacy subdomain needs to be done. The assessment model to assess the development of student 
literacy skills for each domain also needs to be examined, so that the mathematical literacy 
ability of each stage can be measured. Likewise, students’ difficulties in solving problems 
related to mathematical literacy skills also need to be described and planned how to cover it. 
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